1. Field of the Invention
The present invention relates to a moving picture scene detection system for detecting a distinctive scene from moving picture data recorded in a digital recording medium in a compressed form, and for recording the detected scene into the same.
2. Discussion of the Related Art
In editing moving pictures of the cinema or the television, desired scenes are cut out of a sequence of consecutive pictures of films or video tapes and are arranged and connected as intended.
The edit work of editing moving pictures basically consists of the following three procedural steps.
1) To search and pick up desired scenes from among a series of pictures.
2) To cut out these scenes.
3) To arrange and connect the cut-out scenes in an intended sequence.
Of these three steps, step 1) most impedes the edit work for the following reasons.
a) The work to search and pick up the desired scenes is based on the visual inspection. Because of this, much time is consumed for this work.
b) The editor must be highly skilled for the search/pickup work.
Recently, the moving picture coding technique has remarkably advanced. For example, an MPEG (motion picture expert group) coding technique as one of the candidates for the international standards for stored moving picture coding enables the moving pictures for approximately one hour to be recorded in a single CD-ROM. In this situation, new technical problems will arise. For example, economical image data recording media must be produced. A great amount of visual information must be handled. The efficiency of the scene retrieval work of step 1) must be improved.
With the above background, the systems for automatically retrieving desired scenes have been studied and proposed.
For example, Audio has proposed a scene retrieval method based on quantities of features of images in his paper entitled "An interactive motion picture editing", The Institute of Electronics, Information and Communication Engineers, Technical Report, Vol. IE90-6, 1990.
Nagasaka and Tanaka has proposed a method of detecting scene changes in which the scenes of markedly low correlation between the frames are recognized as scene changes, in their paper entitled "Automatic scene-change detection method for video works", Information Processing Society of Japan, 40th (Heisei 2, first half period) National Convention Papers, 1Q-5, pp. 642 to 643, 1990.
Japanese Patent Unexamined Publication No. Hei. 4-219878 discloses a unique scene change detection support technique which uses the magnitude of difference data between the present and preceding frames for scene change detection. This support technique is realized by the arrangement shown in FIG. 13, which includes a scene change check section 801, a moving picture data section 802, an attribute data decoding section 803, a data expanding section 804, and a display section 805. Moving picture data and attribute data, such as data quantity and frame positions, are stored every frame in the moving picture data section 802, in the format as shown in FIG. 14. Let us consider a case where the moving picture data section 802 stores the difference data between the preceding and the present frames as the moving picture data, and the code quantity of each frame and the frame position as the attribute data. In this case, the candidates for scene changes can be detected by comparing the code quantity with a threshold value. The frame of a large code quantity (the quantity of difference data between the preceding and the present frames) indicates such a frame of which the data quantity is greatly different from that of the preceding frame. This fact indicates that a scene change highly probably occurs in this field. The frame in which an moving picture occupies a large area has also a large difference data quantity. The threshold process of the code quantity mistakenly detects this frame as the scene of the scene change. In this respect, the threshold process of the code quantity is not versatile.
Haseyama, Tanaka, and Ohba have proposed a method for retrieving, by using the code quantity of each frame, desired scenes of the moving picture data coded by the interframe predictive DCT coding technique, in their paper entitled "A Study on video retrieval using interframe coded data", The Institute of Electronics, Information and Communication Engineers, 1992 Spring Convention Papers, D-292.
When moving pictures are edited or intended scenes are retrieved from the moving pictures, the whole moving picture data must be played back (or fast fed). The playback of the whole moving picture data consumes much time and makes it impossible to concurrently carry out the edit or the scene retrieval work by using a plurality of pieces of moving picture data. To store the moving picture data not coded, a large memory capacity is required for the memory used. If the moving picture data after coded is stored, another problem arises. To retrieve desired scenes from the stored, coded moving picture data, the whole moving picture data must be decoded. This decoding work consumes much time.
The proposal by Audio in his paper entitled "An interactive motion picture editing", The Institute of Electronics, Information and Communication Engineers, Technical Report, Vol. IE90-6, 1990, and the proposal by Nagasaka and Tanaka in their paper entitled "Automatic scene-change detection method for video works", Information Processing Society of Japan, 40th (Heisei 2, first half period) National Convention Papers, 1Q-5, pp. 642 to 643, 1990, handle the moving picture data not coded. Accordingly, the image data processing systems based on those proposals must have high data processing capabilities. The memory devices used must have large memory capacity.
The technique disclosed in Japanese Patent Unexamined Publication No. Hei. 4-219878, simply checks as to whether or not Data&gt;Th (Data: quantity mount of difference data between the adjacent frames, and Th: threshold value) by the scene change check section 801. Accordingly, the technique is able to detect only one type of scene change. Further, the technique does not record the detected scenes. Therefore, the user must detect the scenes every editing work.
The proposal by Haseyama, Tanaka, and Ohba in their paper entitled "A Study on video retrieval using interframe coded data", The Institute of Electronics, Information and Communication Engineers, 1992 Spring Convention Papers, D-292, retrieves the scenes on the basis of only the code quantity of each frame. In the coding method where the code quantity is controlled so as to keep the quantity of the generated data constant, this proposal cannot provide a high accurate scene detection.